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•- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period wilt apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )^ Responsive to communication(s) filed on 25 June 2003 . 
2a)S This action is FINAL. 2b)D This action is non-final. 

3) 0 Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) ^ Claim(s) 7-36 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 1-36 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) [3 The drawing(s) filed on 20 March 2001 is/are: a)D accepted or b)[3 objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be heldjn abeyance. See 37 CFR 1.85(a). 

11) D The proposed drawing correction filed on is: a)D approved b)Q disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) 0 The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 119 and 120 

13) Q Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)D All b)D Some*c)D None of: 

1 0 Certified copies of the priority documents have been received. 

2.0 Certified copies of the priority documents have been received in Application No. . 



3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 1 7.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 1 9(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 
Attachment(s) 

1 ) £3 Notice of References Cited (PTO-892) 4) O Interview Summary (PTO-41 3) Paper No(s). . 

2) CH Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) EH Notice of Informal Patent Application (PTO-152) 

3) d Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) d Other: 
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DETAILED ACTION 



1 . This office action is responsive to the following communications: Application, 
filed 03/20/2001; Amendment A, filed 06/30/2003 

2. Claims 1-36 are pending in this application. Claims 1,16 and 24 are 
independent claims. No claims have yet been amended. 

3. The present title of the application is "Morph Map Based Real-Time Rendering" 
(as originally filed). 



4. Color photographs and color drawings are acceptable only for examination 
purposes unless a petition filed under 37 CFR 1.84(a)(2) is granted permitting their use 
as acceptable drawings. In the event that applicant wishes to use the drawings 
currently on file as acceptable drawings, a petition must be filed for acceptance of the 
color photographs or color drawings as acceptable drawings. Any such petition must be 
accompanied by the appropriate fee set forth in 37 CFR 1 .17(h), three sets of color 
drawings or color photographs, as appropriate, and an amendment to the first 
paragraph of the brief description of the drawings section of the specification which 
states: 

The patent or application file contains at least one drawing executed in color. Copies of this patent or 
patent application publication with color drawing(s) will be provided by the U.S. Patent and Trademark 
Office upon request and payment of the necessary fee. 

Color photographs will be accepted if the conditions for accepting color drawings 
have been satisfied. 



Drawings 
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Claim Rejections - 35 USC § 102 



5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed 
publication in this or a foreign country, before the invention thereof by the applicant for a patent. 

6. Claims 1-36 rejected under 35 U.S.C. 102(a) as being anticipated by Blanz et al., 
A Morphable Model for the Synthesis of 3D Faces, ACM, 1993, pages 279-288. 

As per independent claim 1 , a method for simulating a real-time rendering . . . 
comprising the steps of: a) precomputing data defining a behavior of light rays 
illuminating the object in regard to the single static viewpoint based on a plurality of 
input images, to produce a plurality of morph maps . . . ; (Blanz discloses illuminations 
conditions can be variables subject to optimization, page 188, section 1.1) b) in 
response to one of a user action and an event that indicates the desired graphical 
effect, performing a transformation two-dimensionally using the plurality of morph maps 
. . . ; (Blanz discloses a set of facial expressions expressed two dimensional for 
morphing, i.e. transformation, page 189, section 3.1, see also figure 4, page 190) and c) 
displaying the output image. (Blanz discloses displaying the illuminated corrected 
morphed image, figure 4, page 190) 

With respect to dependent claim 2, precomputing comprises the step of 
producing data that include a blend factor. (Blanz discloses blending the borders of the 
image data according to an algorithm proposed for images, page 189, section 3) 
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With respect to dependent claim 3, precomputing comprises the step of 
producing data that include an additive factor that is used to control saturation of the 
output image. (Blanz discloses adding to or subtracting vectors, which will manipulate a 
specific attribute, page 189, section 3.1, see also figure 2) 

With respect to dependent claim 4, precomputing comprises the step of tracing 
rays of light to determine the plurality of morph maps . . . (Blanz discloses rendering 
parameters including the intensity of directed light and other light features, page 190, 
section 4) 

With respect to dependent claim 5, a) producing a plurality of warped images . . . 
; and b) combining the plurality of warped images over a range, with a cross-dissolve, to 
produce successive output images . . . (Blanz discloses reconstruction by warping the 
image to generate new orientation, figure 8, page 194) 

With respect to dependent claim 6, performing the transformation comprises the 
step of mapping a selected portion of a surface of the object onto a different part of the 
object to simulate an effect corresponding to movement . . . (Blanz discloses the 
performing morphing for motion-capture data to pictures or 3D face models, page 187, 
section 1) 

With respect to dependent claim 7, only pixels of the object that have been 
altered during the transformation to implement the effect are recomputed in the output 
image. (Blanz discloses that in mapping the facial attributes only attributes that have 
been manipulated will change while other attributes are kept constant, therefore making 
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it inherent that only the pixels in the portion of the face that have been altered will be 
changed, i.e. recomputed, page 189, section 3.1) 

With respect to dependent claim 8, a) providing a grid of cells that overlies and 
bounds pixels . . . ; b) for each cell of the grid, associating an arbitrary rectangle . . . ; 
and c) determining a union of all rectangles . . . (Blanz discloses a segmented 
morphable model, page 189, section 3) 

With respect to dependent claim 9, using an index to map between a region in an 
input image and a corresponding region in the output image, to determine which portion 
of one of the input image and the output image is changed . . . (Blanz discloses 
increasing the expressiveness of the model by dividing the faces into sub regions which 
are morphed independently to define the regions on a reference face, page 189, section 
3) 

With respect to dependent claim 10, a) mapping a texture onto the object in the 
output image; b) applying a reflection to the object . . . ; c) applying a refraction of the 
object . . . (Blanz discloses morphing the image using model parameters such as 
intensity of ambient light, and the intensity of directed light, and further using Phong 
illumination which takes in account the direction of the reflected ray, pages 190-191, 
section 4) 

With respect to dependent claim 1 1 , precomputing includes the step of storing 
anti-aliasing data for use in producing the output image. (Blanz discloses an optic flow 
to generate a smooth interpolation of the vectors coupled with their neighbors, therefore 
providing an algorithm to the user for antialiasing the image, page 192, section 5.1) 
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With respect to dependent claim 12, precomputing is based on one of a three- 
dimensional geometry of the input images and a set of properties of a material in the 
input images. (Blanz discloses morphing based upon shape and texture, see page 189, 
figure 2) 

With respect to dependent claim 13, data produced in the step of precomputing 
includes a lookup table in which parameters used in producing the output image are 
stored. (Blanz discloses the use of a database to store the data, section 2, page 188, 
therefore making it inherent that the data could be stored in other storage means such 
as a look-up table) 

With respect to dependent claims 14 and 15, they are rejected based upon 
similar rational as above independent claim 1 . 

As per independent claim 16, a method for simulating the rendering of graphical 
effects . . . comprising the steps of: a) precomputing a plurality of morph maps of a 
displayed scene . . . ; (Blanz discloses generating morphable face models of a plurality 
of photographs, abstract) b) storing the morph maps for subsequent use in simulating 
rendering of a selected effect . . . ; (Blanz discloses storing the pre-processing scans for 
the morph maps, page 188, section 2) c) transforming at least one input image using a 
blending of the plurality of morph maps . . . ; (Blanz discloses transforming the shape 
and texture of the exampled into a vector space representation and modeling the new 
faces by forming linear combinations, abstract) and d) displaying the output image . . . 
(Blanz discloses displaying the output image, figure 1) 
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With respect to dependent claim 17, a) anti-aliasing to smooth edges in the 
output image; (Blanz discloses an optic flow which generates a smooth interpolation of 
the vectors coupled with their neighbors, therefore providing an algorithm to the user for 
antialiasing the image, page 192, section 5.1) b) displaying light refraction . . . ; c) 
displaying light reflection . . . ; (Blanz discloses on page 193 new illumination for 
morphing an image) d) morphing between an object in the displayed scene and a 
substantially altered object . . . ; (Blanz discloses in figure 4, page 190) and e) 
dynamically warping a selected portion . . . (Blanz discloses in figure 8, page 194) 

With respect to dependent claim 18, a) an index that identifies a pixel data set 
from among a plurality . . . ; b) an image identifier . . . that indicates one of: i) the input 
image . . . ; and ii) a constant color . . . ; c) coordinates of the pixel in the input image; 
(Blanz discloses the morphable model defined by a shape vector which represents the 
geometry that contains the coordinates, and a texture vector that contains the color 
values of the vertices of the image, page 189, section 3) d) the constant color that is to 
be applied to the pixel . . . ; (Blanz discloses a normal distribution to the data based on 
the average shape and texture to generate the differences, page 189, section 3) f) an 
additive factor used to shift the appearance . . . ; (Blanz discloses multiples of the shape 
and texture can be added to or subtracted from any individual face, page 190, section 
3.1 ) and g) a blending factor . . . (Blanz discloses blending the borders of the image 
data according to an algorithm proposed for images, page 189, section 3) 

With respect to dependent claim 19, precomputing comprises the step of 
computing the plurality of morph maps with a light-simulating algorithm . . . (Blanz 
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discloses rendering parameters including the intensity of directed light and other light 
features, page 190, section 4) 

With respect to dependent claim 20, rendering of a textured patch on a surface of 
an object as the patch is dragged over the surface . . . (Blanz discloses computing the 
correspondence in high resolution considering shape and texture data simultaneously 
and making deformations plausible, page 188, section 1.1) 

With respect to dependent claim 21 , rendering of an object simulating a refraction 
that occurs as light reflected from the object passes through a non-homogeneous 
medium . . . (Blanz discloses rendering parameters including the intensity of directed 
light and other light features, page 190, section 4) 

With respect to dependent claim 22, only pixels in the input image that have 
changed are transformed to produce the output image. (Blanz discloses that in 
mapping the facial attributes only attributes that have been manipulated will change 
while other attributes are kept constant, therefore making it inherent that only the pixels 
in the portion of the face that have been altered will be changed, i.e. recomputed, page 
189, section 3.1) 

With respect to dependent claim 23, bi-directionally mapping between each of a 
plurality of pixels in a selected region of the input image and a corresponding pixel in a 
corresponding region of the output image . . . (Blanz discloses in figure 2, page 189 
allowing a user to add to or subtract from a morphable model to generate a deviation in 
the shape and texture, therefore allow for bi-direction mapping between each region) 
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As per independent claim 24 and dependent claims 25-36, they are rejected 
based upon similar rational as above independent claim 1 and dependent claims 2-15 
respectively. 

Response to Arguments 

7. Applicant's arguments with respect to claims 1-36 have been considered but are 
moot in view of the new ground(s) of rejection. 



8. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 



Conclusion 



• # 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Motilewa A. Good-Johnson whose telephone number is 
(703) 305-3939. The examiner can normally be reached on Monday - Friday 8:30 AM - 
5:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mike Razavi can be reached on (703) 305-4713. The fax phone numbers 
for the organization where this application or proceeding is assigned are (703) 872-9314 
for regular communications and (703) 872-9314 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is (703) 306- 
0377. 



mgj 

August 28, 2003 



Motilewa A. Good-Johnson 
Examiner 
Art Unit 2672 
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